Method of taking pictures for generating three-dimensional image data

ABSTRACT

A method of taking pictures for generating three-dimensional image data is disclosed. The method includes continuously illuminating a target object with an environmental light; using an image sensor to receive the light reflected from the target object to generate a first electrical image signal; illuminating the target object with an active light generated from an active light source unit during the period of exposure by the environmental light; using the same image sensor to receive another reflected light to generate a second electrical image signal; using an image processing unit to receive the first and second electrical image signals, convert the first electrical image signal to a two-dimensional image data, and subtract the first electrical image signal from the second electrical image signal to generate three-dimensional depth data; and combining the two-dimensional image data and the three-dimensional depth data to generate the three-dimensional image data.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention generally relates to a method of taking picturesfor generating three-dimensional image data, and more specifically tousing the light generated by the active light source unit to illuminatethe target object while illuminated by the environmental light.

2. The Prior Arts

Generally, the three-dimensional camera system consists of twoindependent two-dimensional cameras which are configured by a specificdisplacement pitch, faced towards the target object, and taking picturesso as to simultaneously generate two independent images, similar to thevision mechanism of human eyes. That is, the parallax informationthrough the calculation of the independent images from these two planecameras is similar to the visional depth perceived by human eyes. Then,these two independent images can be stored in the suitable storagemedium, such as magnetic disk, movie tape or electronic memory.

However, the above-mentioned method in the prior arts for generating thethree-dimensional image usually needs certain mechanical structure withvery precise standard for mass production. Otherwise, the problem ofdepth map possibly results from the error of the relative positions andangles for the cameras. The method for generating the three-dimensionalimage is primarily based on the specific calculation related to rightand left eyes, and it thus leads to high cost of calculation and needssome high performance CPU (central processing unit) to implement.

Therefore, a method of taking pictures for generating three-dimensionalimage data with only one camera to eliminate the requirement of highprecise mechanic structure in mass production has been greatly desired.The cost of calculation is reduced, and the output of the depth image aswell as the color image is thus obtained.

SUMMARY OF THE INVENTION

A primary objective of the present invention is to provide a method oftaking pictures for generating three-dimensional image data through anactive light source unit, an optical unit, an image sensor, and an imageprocessing unit.

The method in accordance with the present invention comprisescontinuously illuminating a target object with an environmental light;using an image sensor to receive the light reflected from the targetobject to generate a first electrical image signal; using an activelight generated by an active light source unit to illuminate the targetobject while illuminated by the environmental light; using the sameimage sensor to receive another reflected light from the target objectfor the active light to generate a second electrical image signal,wherein the exposure interval for the image sensor is the exposureinterval for the second electric image signal; using an image processingunit to receive the first and second electrical image signals andperform image processing by converting the first electrical image signalto a two-dimensional image data, and subtracting the first electricalimage signal from the second electrical image signal to generate athree-dimensional depth signal which is converted to a three-dimensionaldepth data; and combining the two-dimensional image data and thethree-dimensional depth data to generate the three-dimensional imagedata. Finally, an output having a depth image together with a colorimage is thus obtained.

The algorithm in the method of the present invention is implemented forfast switched active light source units, such as light-emitting diodes(LEDs), instead of slowly switched light source units, like conventionallight bulbs. The so-called “fast switched” is usually determined by theframe rate of the image sensor. For example, if the frame rate is set 30FPS (frame per second), it usually takes about 33 ms for each frame ofpicture, and the switching rate of the active light source unit has tobe faster than that of the period of 33 ms so as to assure that theamount of exposure for the image sensor is sufficient. Therefore, theswitching operation of the active light source unit has to besynchronized with the start and end points of time for the exposure ofthe image sensor, like the vertical synchronized signal. The real-timeproperty and the capability of noise resistance for the output of depthdata are improved and increased.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention can be understood in more detail by reading thesubsequent detailed description in conjunction with the examples andreferences made to the accompanying drawings, wherein:

FIG. 1 is a flow chart to illustrate the processes of the methodaccording to the present invention; and

FIG. 2 shows a system diagram to illustrate the method according to thepresent invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

The present invention may be embodied in various forms and the detailsof the preferred embodiments of the present invention will be describedin the subsequent content with reference to the accompanying drawings.The drawings (not to scale) show and depict only the preferredembodiments of the invention and shall not be considered as limitationsto the scope of the present invention. Modifications of the shape of thepresent invention shall too be considered to be within the spirit of thepresent invention.

Please refer to FIGS. 1 and 2, showing the flow chart and the schematicsystem diagram for the method of taking pictures for generatingthree-dimensional image data according to the present invention,respectively. As shown in FIG. 1, the method of the present inventionfirst starts in the step S10 by illuminating the target object 20 withthe environmental light 10 to generate the first image reflected lightreflected from the target object 20.

Then, in the step S20, the first image reflected light is incident ontothe image sensor 40 through the optical unit 30, which may comprisetransparent optical glass or lens. The image sensor 40 may comprise aconventional a CMOS or CCD (charge coupled device). Next, the imagesensor 40 receives the first image reflected light and generate thefirst electrical image signal in the step S30.

The step S40 is performed later by using the active light source unit 50to generate the active light 60 illuminating the target object 20, whichis continuously illuminated by the environmental light 10 during theexposure (based on the synchronized signal for the exposure). The activelight 60 is reflected by the target object 20 to form the second imagereflected light. The active light 60 has a specific range of wavelengthwhich can pass through the optical unit 30 or be filtered by the opticalunit 30 such that the capability of noise resistance at a long distanceoutdoors increases. Moreover, it is preferred to use the active lightsource unit 50 with lower power consumption.

The above active light source unit 50 may comprise light emitting diodes(LEDs) or laser, and the active light 60 has a specific range ofwavelength, covering at least one of visible, ultraviolet, and infraredlight. The flickering frequency of the active light 60 is less than theframe rate (such as frame per second (FPS)) of the image sensor 40, andthe flickering frequency is maintained as a constant ratio which can beset as desired. The range of the constant ratio can be set as any realnumber between 0 and 1, but not 0 or 1. After the constant ratio is set,the integral of intensity of the active light 50 within a second issubstantially not changed, and proportional to the constant ratio. Ifthe constant ratio is set higher than 0.5 and the image sensor 40 with aframe rate of 60 FPS is installed, the active light 60 generated by theactive light source unit 50 should have the flickering frequency morethan 30 per second. It is primarily intended that human eyes can notperceive the flicker in the active light 60 when the active light sourceunit 50 able to generate visible light is used. Further, if the activelight source unit 50 generating infrared light and the image sensor 40with 30 FPS are employed, the constant ratio can be set less than 0.5 tocause the flickering frequency less than 15 per second such that theflicker induced is also not perceivable to human eyes because infraredlight is substantially invisible. However, the shortcoming is that theproperty of the real time is slower than the former by one time. If bothfeatures should be met, it is preferred to use the image sensor 40 witha frame rate higher than 60 FPS, and the active light source unit 50with invisible light, such as infrared light.

Furthermore, the switching on/off of the active light source unit 50 isbased on the synchronized signal related to the exposure of the imagesensor 40 to derive the suitable points of time to start and end so asto meet the image sensor 40 with different specifications and designs.As shown in FIG. 2, the active light source unit receives thesynchronized signal transmitted from the image sensor.

The intensity of the environmental light 10 can be sustained without anychange within a variable interval of time, but vary out of the variableinterval of time. Specifically, the intensity of the environmental light10 has a variable frequency. The active time interval of exposure forthe active light source unit 50 is less than the variable interval oftime for the environmental light 10. That is, the flickering frequencyof the active light 60 should be higher or equal the variable frequencyof the intensity of the environmental light 10.

Later, the step S50 is performed. The second reflected light is incidentonto the image sensor 40 through the optical unit 30, and the secondelectric image signal is thus generated by the image sensor 40.

Next, in the step S60, the image processing unit 70 is used to receivethe first and second electric image signals, and the first electricimage signal is converted to the two-dimensional plane image data, andthe second electric image signal is subtracted by the first electricimage signal to form the three-dimensional electric depth signal, whichis further converted to the three-dimensional depth data. Thetwo-dimensional plane image data represents the plane image sensed bythe image sensor 40 for the target object 20. The plane image comprisesa plurality of unit images, which can be color or gray leveltwo-dimensional images dependent on the type of the image sensor 40used. The three-dimensional depth data represents the depth and therelative depth for each unit image and the image sensor 40. Themagnitude of the three-dimensional electric depth signal is inverselyproportional to the relative depth by N powers of exponent, and N is anyreal number between 1 and 10. Specifically, as larger N is, the fasterthe three-dimensional electric depth signal is reduced as the relativedepth increases. N can be set as actually desired. As described above,the active light source unit 50 is designed with a specific range ofwavelength, covering at least one of visible, ultraviolet and infraredlight. It may be worth noting that the three dimensional depth data isdetermined by the magnitude of the three-dimensional electric depthsignal regardless of the specific range of wavelength.

Finally, in the step S70, the image processing unit 70 combines thetwo-dimensional plane image data and the three-dimensional depth data toform the three-dimensional image data for the target object 20.

Moreover, the active light source unit 50 can be further controlled bythe image processing unit 70 to generate the active light 60, whichactively illuminates the target object 20 in the active exposureinterval. Also, the image processing unit 70 is used to accomplish thethree-dimensional image data between two intervals of time for theactive exposure. Therefore, it is possible to provide successivethree-dimensional image data.

Additionally, in the step S30, it is necessary for the intensity of theactive light 60 to take into consideration the configuration of therelative working distance (of the objects, light sources and camera) andthe first electric image signal so as to avoid being too close to theupper limit of the brightness for the pixels of the image sensor. If thequality of color image is deteriorated, the post image treatment isneeded to improve the quality of the output image according to the imagequality desired. Therefore, it is possible to obtain both the depthimage and the color image according to the features provided by abovementioned embodiments.

Although the present invention has been described with reference to thepreferred embodiments, it will be understood that the invention is notlimited to the details described thereof. Various substitutions andmodifications have been suggested in the foregoing description, andothers will occur to those of ordinary skill in the art. Therefore, allsuch substitutions and modifications are intended to be embraced withinthe scope of the invention as defined in the appended claims.

What is claimed is:
 1. A method of taking pictures for generatingthree-dimensional image data, comprising steps of: illuminating a targetobject with an environmental light to generate a first image reflectedlight reflected from said target object; using a color image sensor toreceive said first image reflected light through a non-movable opticalunit; generating a first electrical image signal by using said imagesensor based on said first image reflected light, the first image is acomplete frame; using an active light source unit to generate an activelight to illuminate said target object while said object is continuouslyilluminated by said environmental light, wherein said active lightsource unit receives a synchronized signal from said image sensor and isswitched on/off based on the synchronized signal, and said target objectreflects the illuminating lights to form a second image reflected light,and wherein said image sensor includes an pixel array and the built-incontroller, and the built-in controller of said image sensor generatesthe synchronized signal, also said active light source unit issynchronized with the start and end points of time for the exposure ofthe image sensor, and said synchronized signal is a verticalsynchronized signal; using said image sensor through said optical unitto receive said second image reflected light including both reflectedlight from said active light and reflected light from said environmentallight and generate a second electrical image signal, the second image isa complete frame; transporting said first and second electrical imagesignals stored in a memory, wherein said first and second electricalimage signals are digital signals; using an image processing unit toreceive said first and second electrical image signals, convert saidfirst electrical image signal to a two-dimensional image data, andsubtract said first electrical image signal from said second electricalimage signal to generate a three-dimensional electrical depth signal,wherein said three-dimensional electrical depth signal is converted tothree-dimensional depth data, wherein said two-dimensional image datarepresents a plane image of said target object sensed by said imagesensor, said plane image comprises a plurality of unit images which aretwo-dimensional images with colors, and said three-dimensional depthdata represents a depth or a relative depth between said image sensorand each of said unit images; and combining said two-dimensional imagedata and said three-dimensional depth data such that finally an outputhaving a depth image together with a color image is thus obtained,wherein said active light unit has a flickering frequency less than aframe rate of said image sensor with a constant ratio specified asdesired, said constant ratio being a real number from 0 to 1, but not 0or 1, an integral of intensity of said active light within a second issubstantially not changed after said constant ratio is specified, andsaid integral is proportional to said constant ratio; and wherein saidactive light includes a specific range of wavelength, and saidthree-dimensional electrical depth signal is converted to saidthree-dimensional depth data according to magnitude of saidthree-dimensional electrical depth signal independent of the specificrange of wavelength, and the magnitude of said three-dimensionalelectrical depth signal is inversely proportional to said relative depthby N powers of exponent, N being 2, 4, 5, 6, 7, 8 or
 9. 2. The method asclaimed in claim 1, wherein said optical unit comprises an optical glassor lens, and said active light has a specific range of wavelengthadapted to said optical unit to pass or be filtered.
 3. The method asclaimed in claim 1, wherein said active light comprises laser or lightgenerated by light emitting diode (LED) with wavelength within aspecific range, comprising at least one of visible, ultraviolet, andinfrared light.
 4. The method as claimed in claim 1, wherein said activelight source unit is switched on/off according to appropriate start andend points of time to meet different specification and design for saidimage sensor, and said start and end points of time are derived from asynchronized signal related to said exposure of said image sensor. 5.The method as claimed in claim 1, wherein said active light source unitis controlled by said image processing unit to generate said activelight.
 6. The method as claimed in claim 1, wherein said active lightsource unit generates said active light according to said image sensor.7. The method as claimed in claim 1, wherein said active light is avisible laser or light generated by light emitting diode (LED) withwavelength within a specific range.
 8. The method as claimed in claim 1,wherein said active light is an ultraviolet laser or light generated bylight emitting diode (LED) with wavelength within a specific range.